It looks like within the Functional Testing suite the playwright test headless command is displaying this error:

1. testing/tests/headless.sitesearch.spec.js:10:3 › Site search submit the autocomplete search form will redirect to site search .. Timeout of 30000ms exceeded.

It looks like within the Functional Testing suite the playwright test headless command is displaying this error:

1. testing/tests/headless.sitesearch.spec.js:10:3 › Site search submit the autocomplete search form will redirect to site search .. Timeout of 30000ms exceeded.

Looks like that was a hiccup. I reran all the actions and everything looks to be green now.

@tannerdolby I cannot remember. With this PR, did you manage to address the issue that we ran into the last time we merged this work, or is that still on the to-do list? Thanks!

@schalkneethling No problem! I've looked through the proposed changes here and couldn't pinpoint exactly what caused the issue when it was previously merged. If we can figure out how to create a reproducible example of that error then the problematic code should hopefully become more clear to us.

@schalkneethling No problem! I've looked through the proposed changes here and couldn't pinpoint exactly what caused the issue when it was previously merged. If we can figure out how to create a reproducible example of that error then the problematic code should hopefully become more clear to us.

@escattone Thoughts?

@tannerdolby Thanks for all of your work to date on this! Unfortunately, the engineering team doesn't have much time to look into this further given other priorities. If you have the time and desire, for next steps I would recommend:

1. Running the build within this branch on the translated content to identify one or more documents that fail.
2. Selecting one of those documents as your test case to use for debugging the root cause.

You're welcome @escattone! I have the time/desire to see this through so I'm happy to stick around until we pinpoint the root cause. I've done the steps you suggested and learned a few things.

When running a full build on translated content within this branch, the first translated content page to raise an error is:

/Users/TannerDolby/translated-content/files/de/learn/html/introduction_to_html/advanced_text_formatting/index.html

Error: MacroLiveSampleError within /Users/TannerDolby/translated-content/files/de/learn/html/introduction_to_html/advanced_text_formatting/index.html, line 145 column 4 (unable to find any live code samples for "Playable_code_1" within /de/docs/Learn/HTML/Introduction_to_HTML/Advanced_text_formatting)

and for the remainder of the build many errors are logged but none of the pages actual fail and cause the build to halt.

Inside of index.js where this MacroLiveSampleError flaw is handled, if (flaw.name === "MacroLiveSampleError") { ... there is a comment mentioning,

// As of April 2021 there are 0 pages in mdn/content that trigger
// a MacroLiveSampleError. So we can be a lot more strict with en-US
// until the translated-content has had a chance to clean up all
// their live sample errors.
// See https://github.com/mdn/yari/issues/2489

As the only major errors when running a build on translated content are MacroLiveSampleErrors, it leads me to believe that this is probably where the error that broke the build might be coming from.

if (document.metadata.locale === "en-US") {
          throw new Error(
            `MacroLiveSampleError within ${flaw.filepath}, line ${flaw.line} column ${flaw.column} (${flaw.error.message})`
          );
        } else {
          console.warn(
            `MacroLiveSampleError within ${flaw.filepath}, line ${flaw.line} column ${flaw.column} (${flaw.error.message})`
          );
        }

In this comment, Peter mentions, "So now, for en-US, it will crash the CI build, for translated-content it will just be a console.warn". Which is what I'm seeing, it will just be a console.warn (not an error that will crash the CI build) which I'm receiving in the logs, I just can't reproduce the CI build crashing. Since I didn't see any MacroLiveSampleErrors for en-US pages and only for translated-content, it led me to believe the throw new Error() was never being called since that would produce an error that crashes the build. I'm not entirely sure if this is conflicting the with logic for handling headings in this PR but it seemed worthwhile to learn more about.

There were other errors (aside from the MacroLiveSampleError) in the build for content and translated content like:

• In CSS_Ref the smartLink to /es/docs/Web/CSS/transform-function/rotateX is broken (many of these)
• Images missing src attributes: In /es/docs/Learn/Server-side/Configuring_server_MIME_types there's an img tag without src (<img align="right" alt="Example of an incorrect MIME type result">)
• Can't get English original from fr/conflicting/learn/css/building_blocks/styling_tables
• Unable to decodeURI 'https://www.ecma-international.org/ecma-262/6.0/#sec-get-%typedarray%.prototype.buffer'. Will proceed without.

@schalkneethling I will use one of these first few translated pages as a test case and scan through my proposed changes in this PR to better figure out what could be causing problems for translated-content. I will do some further investigating.

Thanks so much @tannerdolby. Much appreciated.

@caugner Thanks for review so far. I've added the suggestions you had and everything is running now. The only problem is that functional testing is failing and<h4> elements don't have the expected <a> tags, the test cases for h4s are only picking up the text content when .html() is asserted against so the links arent being generated.

Apologies for all the testing/debugging commits in this last batch. The yarn test:testing wasn't producing the live results when running locally so I was relying on the build & testing pipeline logs here on GitHub.

Expected: "<a href=\"#final_heading\" title=\"Permalink to Final heading\">Final heading</a>"
    Received: "Final heading"

1805 |   expect(h4.html()).toBe(
     |                    ^
1806 |     '<a href="#final_heading" title="Permalink to Final heading">Final heading</a>'
1807 |   );

I haven't been in the codebase in awhile so I'm still getting familiarized with things again, but I could imagine the document-extractor.js just needs to be updated to implement similar logic as the h2/h3s within the _addSectionProse function. I tried experimenting but wasn't entirely sure how the whole flow worked. It might be something straightforward that I'm not seeing.

I had to add isH4 as a property to the types in document.ts that could be assigned to the Section type. This was for resolving the errors that were being thrown to get things running but it shouldn't have any destructive behavior.

FWIW We need to make sure that doesn't break the EmbedLiveSample macro (see this issue).

This pull request has merge conflicts that must be resolved before it can be merged.